Analyzing historical I/O accesses to predict future system behavior may be prohibited by the volume of I/O accesses in a computer system. The overhead in processing time added to every I/O to log the data and the amount of space needed to store the access data may be prohibitively expensive. As an example, a 96 TB storage system with 8 KB blocks has 12 billion blocks. Even supposing that 10% are regularly accessed, that is 1.2 billion blocks to track. Tracking access patterns to billions of blocks can easily consume GB of memory and/or billions of disk accesses.
For that reason, most systems do not log I/O accesses at all. Others may log at a coarser granularity, recording representative samples of access. Still other may log only a limited time horizon of history such as a few hours, or tracking access to very large block sizes. However, short term or coarse-grained access histories may not enable the discovery of important access patterns that may be useful for predicting future system behavior.
A technical challenge to using historical access data to make more efficient cache and storage management decisions is how to maintain a detailed history of I/O accesses given limited memory and I/O availability without adversely impacting the performance of the system. The process that tracks I/O patterns may not interfere with client storage accesses. A solution is needed that requires minimal resources.